Build an AI API Billing Ledger: Quote, Reserve, Settle, and Reconcile Every Model Call

Customer-facing AI API billing cannot be a monthly export of raw provider usage. If a gateway exposes multiple models to tenants, teams, or partners, billing has to answer a harder question before the invoice exists: should this request be allowed right now, and how will its cost be explained later?

The practical pattern is a billing ledger with four stages: quote, reserve, settle, and reconcile. Quote the likely cost before the request. Reserve enough tenant budget to cover the allowed worst case. Settle the actual cost after usage is known. Reconcile the gateway ledger against provider-side records so invoices remain defensible.

This article describes that control loop for a multi-model API gateway. It is useful whether the gateway bills internal teams, prepaid customers, agency clients, or downstream partners.

The billing problem: provider usage is not a customer invoice

Fact: major AI providers do not expose one universal token counter or one universal price. OpenAI publishes per-model prices with separate input, cached input, and output token rates. OpenAI prompt caching reports cached token usage in the API response usage field. Anthropic documents separate counters for normal input tokens, cache creation input tokens, cache read input tokens, and output tokens. Gemini pricing distinguishes input, output, and other token categories, including modality-specific usage such as audio tokens.

That means a gateway cannot safely bill by multiplying total_tokens by one price. It needs provider-specific adapters behind a provider-neutral billing schema.

The problem becomes more visible in these situations:

• Prepaid credits: the gateway must reject requests before the tenant spends below zero.
• Partner markups: the partner needs its own customer-facing invoice, not a copy of the provider bill.
• Streaming: the response begins before final token usage is known.
• Prompt caching: cached input may be cheaper than uncached input, but only if measured separately.
• Reasoning and tool use: some models expose additional usage dimensions, hidden output classes, or media units.
• Provider price changes: an invoice from last month must still be reproducible after a rate card changes.

Recommendation: treat billing as an append-only financial ledger, not as a dashboard query over request logs.

The core architecture

A reliable billing architecture has six components:

1. Tenant account: customer, workspace, reseller client, or internal cost center.
2. Rate-card service: versioned prices for provider, model, billing class, currency, and markup rule.
3. Estimator: calculates a preflight quote from request parameters and model policy.
4. Reservation ledger: holds budget before the provider call starts.
5. Usage normalizer: converts provider-specific usage fields into internal billing units.
6. Settlement and reconciliation jobs: finalize charges and compare them with provider-side records.

The control flow looks like this:

client request
  -> authenticate tenant and key
  -> select model and rate-card version
  -> estimate input and max output cost
  -> reserve tenant balance
  -> call provider
  -> normalize returned usage
  -> settle actual cost
  -> release unused reservation
  -> emit invoice-ready ledger event

The important design choice is that the request is not merely observed. It is financially controlled before and after execution.

Step 1: quote before the provider call

A preflight quote should be pessimistic enough to enforce budgets but explainable enough to show to customers or partners.

Inputs usually include:

• tenant ID and billing plan;
• API key ID or project ID;
• provider and model ID after routing rules are applied;
• estimated uncached input tokens;
• known cached-input eligibility, if available;
• max_tokens, max_output_tokens, or equivalent output cap;
• tool, image, audio, or other modality parameters;
• partner markup, discount, or reseller pricing rule;
• currency and rounding policy.

A simple quote formula for text generation might be:

estimated_cost =
  estimated_uncached_input_tokens * input_rate
+ estimated_cached_input_tokens   * cached_input_rate
+ max_output_tokens               * output_rate
+ request_fee
+ partner_markup

Recommendation: when final output length is unknown, reserve against the configured maximum output. If the application leaves the output cap unbounded, the gateway should apply a tenant or model default. Budget enforcement cannot be deterministic if there is no maximum liability.

This can reject some requests that would have been cheap in practice. That is the trade-off. For prepaid systems, the safer default is pessimistic reservation with unused funds released after settlement. For invoiced enterprise customers, teams may allow soft overages and use the quote mainly for alerts.

Step 2: reserve tenant budget

The reservation protects the tenant account from spending more than the allowed balance. It should be atomic: either the reservation succeeds and the provider call may start, or the request is rejected before any provider cost is incurred.

A reservation record might include:

{
  "reservation_id": "res_01J...",
  "tenant_id": "tenant_123",
  "api_key_id": "key_456",
  "request_id": "req_789",
  "provider": "example_provider",
  "model": "model-a",
  "rate_card_version": "2026-08-01",
  "quoted_amount": "0.032100",
  "currency": "USD",
  "status": "reserved",
  "expires_at": "2026-08-11T12:05:00Z"
}

Use short reservation expirations for network failures and client disconnects. A cleanup job should release expired reservations that never reached settlement. However, do not release a reservation simply because the client disconnected; the provider call might still complete and incur cost. Track provider request state separately.

Recommendation: make reservation idempotent by request ID or idempotency key. Retries from clients, gateways, or workers should not create multiple budget holds for the same logical request.

Step 3: normalize provider usage

Provider responses should be converted into a small internal schema. Keep it stable even as providers add new usage fields.

A practical normalized usage schema:

{
  "input_uncached_tokens": 1200,
  "input_cached_tokens": 800,
  "cache_write_tokens": 0,
  "output_tokens": 650,
  "reasoning_or_hidden_output_tokens": 0,
  "tool_or_media_units": [],
  "request_fee_units": 1,
  "provider_request_id": "prov_abc",
  "usage_source": "provider_response",
  "is_estimated": false
}

This schema is intentionally not identical to any one provider’s response. It captures the billing dimensions that invoices need while preserving escape hatches for provider-specific units.

Cached tokens need their own line

Fact: prompt caching can be priced differently from uncached input. If cached tokens are merged into total input tokens, the customer may be overcharged or the gateway may understate provider cost. Cached input should appear as its own billing class in both the ledger and the invoice.

Cache writes and cache reads are not always the same

Some providers distinguish between creating cache entries and reading from cache. The normalizer should not assume that cached input always means one billing rate. If a provider has cache-write tokens and cache-read tokens, map them separately or preserve them as provider-specific subunits.

Reasoning and hidden output need a policy

Some models expose reasoning-related usage or hidden output counters. If the provider bills for those units, the gateway must decide whether to show them directly, roll them into an output category, or list them as a separate invoice line.

Recommendation: customer-facing invoices should use plain language. For example: “reasoning output tokens” is clearer than a raw provider field name. Keep raw fields available for audit, but do not force every customer to understand provider internals.

Step 4: settle actual cost

Settlement converts normalized usage into final ledger entries. It should be append-only and reference the rate-card version used for the request.

A settled event might look like this:

{
  "ledger_event_id": "led_01J...",
  "event_type": "settlement",
  "tenant_id": "tenant_123",
  "request_id": "req_789",
  "reservation_id": "res_01J...",
  "provider": "example_provider",
  "model": "model-a",
  "rate_card_version": "2026-08-01",
  "lines": [
    {
      "billing_class": "input_uncached_tokens",
      "quantity": 1200,
      "unit": "token",
      "unit_price": "0.00000250",
      "amount": "0.003000"
    },
    {
      "billing_class": "input_cached_tokens",
      "quantity": 800,
      "unit": "token",
      "unit_price": "0.00000125",
      "amount": "0.001000"
    },
    {
      "billing_class": "output_tokens",
      "quantity": 650,
      "unit": "token",
      "unit_price": "0.00001000",
      "amount": "0.006500"
    }
  ],
  "total_amount": "0.010500",
  "currency": "USD",
  "status": "settled"
}

If the request was reserved for 0.032100 and settled at 0.010500, the ledger releases 0.021600 back to available balance.

Recommendation: never recalculate old invoice lines from the current pricing table. Store immutable rate-card versions and attach the version ID to every quote, reservation, and settlement event. Otherwise, an invoice may become impossible to reproduce after a provider updates model prices.

Streaming requests: reserve first, settle later

Streaming complicates billing because the user begins receiving output before the gateway knows final usage. The answer is not to skip preflight checks. The gateway should reserve before opening the stream.

Use this workflow:

1. Estimate input tokens and maximum output cost.
2. Reserve tenant budget.
3. Open the provider stream.
4. Forward chunks to the client.
5. Capture final usage when the provider sends it or when a follow-up usage record is available.
6. Settle actual cost and release unused reservation.

If final usage is unavailable, mark the settlement as estimated rather than pretending it is exact:

"usage_source": "gateway_estimate",
"is_estimated": true,
"reconciliation_status": "pending"

Recommendation: daily reconciliation should prioritize estimated streaming events, failed requests, timeouts, and retries. These are the areas most likely to create variance between gateway records and provider invoices.

Rate-card versioning and markup rules

A rate card should be a versioned object, not a mutable spreadsheet.

Minimum fields:

• provider;
• model ID;
• billing class;
• unit, such as token, request, image, audio second, or tool unit;
• unit price;
• currency;
• effective start and end timestamps;
• rounding policy;
• tenant plan or partner markup rule;
• source reference and approval metadata.

Markup rules should be explicit. For example:

• Cost plus: provider cost plus 20%.
• Fixed retail: tenant pays a fixed token price regardless of provider price.
• Tiered: first 10 million tokens at one rate, then a lower rate.
• Included credits: usage burns down a monthly allowance before overage billing starts.

Trade-off: rate-card versioning adds operational work, but it prevents invoice disputes from becoming archaeology. A customer support agent should be able to explain why a request on August 3 was billed at a specific rate without checking today’s provider pricing.

Separate the billing ledger from analytics

Analytics and billing have different tolerances. Analytics can be aggregated, delayed, sampled, or corrected. Billing must be complete, idempotent, auditable, and explainable.

Use analytics for questions like:

• Which teams are using the most tokens?
• Which models are growing fastest?
• Where can prompt caching reduce cost?
• Which keys produce unusually expensive requests?

Use the billing ledger for questions like:

• Was this request authorized against the tenant’s balance?
• Which rate-card version produced this charge?
• Was unused reservation released?
• Does the customer invoice match settled usage?
• Does gateway usage match provider-side usage?

Fact: OpenTelemetry GenAI semantic conventions include token usage attributes such as input and output tokens. That is useful for observability and joining traces to cost events. But telemetry attributes are not a substitute for rate cards, reservations, settlement, rounding, and invoice state.

Daily reconciliation workflow

Reconciliation compares the gateway’s settled ledger with provider-side usage. The goal is not perfect agreement on every intermediate field. The goal is to detect material variance early enough to correct invoices, rate cards, or adapters.

A practical daily job:

1. Group gateway ledger events by provider, model, tenant or API key, billing class, and UTC day.
2. Fetch provider-side usage grouped by available dimensions, such as API key ID, model, and day.
3. Normalize provider exports through the same adapter code used for request responses where possible.
4. Compare quantities and costs by billing class.
5. Flag variance above thresholds, such as 0.5% quantity difference or any large absolute cost difference.
6. Classify variance causes: streaming estimates, retries, failed requests, cache accounting, model alias changes, delayed provider records, or missing request IDs.
7. Create adjustment events instead of editing old settlement events.

Recommendation: use provider API keys per tenant where operationally feasible because it simplifies reconciliation. If that creates too much key-management overhead, map internal tenant IDs to provider metadata where supported and keep a reliable request ID bridge.

Invoice lines customers can understand

A customer-facing invoice should not mirror provider JSON. It should explain the bill in stable business terms.

Useful invoice columns:

• date range;
• tenant, project, or API key label;
• model or model profile;
• request count;
• uncached input tokens;
• cached input tokens;
• output tokens;
• media or tool units, if applicable;
• discounts, credits, or markups;
• total amount and currency.

For partners, include both wholesale cost and retail charge only if the business model requires it. Many reseller invoices should show retail usage only, while partner dashboards may show margin separately.

Trade-off: a unified invoice schema improves readability, but provider-specific billing details still need escape hatches. Keep invoice lines simple by default and provide an export for advanced customers who need detailed audit fields.

Implementation checklist

Before launch

• Define normalized billing classes for all supported providers.
• Create immutable rate-card versions with effective dates.
• Require output caps or apply gateway defaults.
• Implement atomic reservations with idempotency keys.
• Set rounding rules for each currency.
• Decide how to invoice cached tokens, reasoning tokens, media units, and request fees.
• Test retries, timeouts, client disconnects, and provider errors.
• Build an adjustment-event mechanism instead of editing settled events.

During request handling

• Authenticate tenant and key.
• Resolve final model after routing and fallback policy.
• Select the correct rate-card version.
• Quote worst-case cost.
• Reserve balance or reject the request.
• Record provider request ID when available.
• Normalize usage from the response.
• Settle, release unused reservation, and emit invoice-ready events.

After request handling

• Run daily reconciliation by provider, key, model, billing class, and day.
• Review estimated streaming settlements.
• Flag model usage with missing rate-card entries.
• Monitor variance caused by cached-token accounting.
• Generate customer invoice previews before final billing.

Predictions to plan for

Prediction: AI API billing will become more multi-dimensional, not less. Token classes, cache classes, media units, tool execution, and reasoning-related counters are likely to keep expanding as model capabilities change.

Prediction: customers will expect usage explanations at the request, key, project, and invoice level. A monthly total without traceable line items will be insufficient for teams reselling API access or enforcing prepaid budgets.

Prediction: gateways that already separate quote, reservation, settlement, and reconciliation will adapt faster to new pricing models because they can add billing classes without rewriting the entire invoice system.

Actionable conclusion

If you expose multiple AI providers through one gateway, build the billing ledger before billing disputes force the issue. Start with four guarantees:

1. Every billable request receives a preflight quote.
2. Every prepaid or capped tenant has budget reserved before the provider call starts.
3. Every provider response is normalized into stable billing classes.
4. Every invoice can be reconciled against provider-side usage and the exact rate-card version used at the time.

That control loop makes unified AI API billing understandable for customers, enforceable for prepaid credits, flexible for partner markups, and auditable when provider pricing or usage formats change.

Related reading

• joining LLM traces with usage and cost records
• building reseller portals with tenant usage metering
• using prompt caching while keeping billing understandable